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•  SHARED  information  (SI): 

-  Information  items  that  are  held  by  all  participants  in  a 
group  decision  making  environment. 

•  CNN  reports,  products  from  common  search  engines,  etc. 

•  UNIQUE  information  (UP: 

-  Information  items  that  are  held  by  one  (or  some,  but  not 
all)  member(s)  of  the  group. 

•  Personal  information  sources,  previous  experiences,  special 
database  access,  etc. 
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Literature  Review  Indicates: 


-  People  are  not  very  good  at  sharing  UI, 
•  Discussion  time  is  spent  mostly  on  SI. 


When  UI  is  shared  with  the  group,  it  often  has 
little  impact  on  final  decision  selection. 

•  Weighting  is  placed  on  SI 
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Classic  Study  (Stasser  &  Titus,  1985): 

Who  Should  We  Admit  to  Our  College? 


Systems  Center 
San  Dieqo 


Condition  1:  Everyone 
has  all  the  information 
(all  SI,  no  UI) 


Condition  2:  all  info  is  on 
table,  but  distributed 
across  people  (some  SI, 
some  UI) 


Probability  of  “correct”  decision. 

I  .83 
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Discussion  of  SI  vs.  UI 
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SI  discussion  “crowds  out”  time  available  for  UI 


of  items 
discussed 


Shared  Unique 

Information  Type 

Very  Consistent  Finding 
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•  In  asynchronous  collaboration,  social 
factors  (prestige,  personality,  etc.)  still  play 
a  role 

BUT.... 

•  the  primary  obstacle  to  effective 
information  exchange  is  the  cognitive 

burden  placed  upon  both  the  sender  and 
recipient  of  the  information. 
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Cognitive  Burden 


•  Is  this  item  really  unique,  and  not  available 
to  the  other  participants? 

•  Is  it  important  enough  to  send? 

•  Is  this  the  best  example  to  send? 

•  Should  I  send  it  now  or  await  for  further 
input/ confirmation? 

•  Send  it  to  all  participants,  or  just  selected 
subgroup? 
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Cognitive  Burden  (receiver  perspective) 
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•  I  have  arriving  information  items  (which  I  did  not 
develop/find)  related  to  my  decision  task.  I  need  to 
determine: 

-  Which  particular  decision  aspect  do  these  items  impact? 

•  Cost,  Risk,  Duration,  etc? 

-  What  is  it’s  potential  influence? 

•  Does  it  have  a  positive  or  negative  influence  on  cost,  risk,  etc.? 

-  Is  this  timely  information  from  a  credible  source? 

•  How  recent  is  this  information?  What  is  the  source? 

-  Does  the  information  impact  the  credibility  of 
information  items  that  I  am  currently  using? 

•  If  so,  what  is  the  overall  “domino  effect”  of  this? 
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•  Overall: 

-  Enhance  the  ability  of  asynchronously  distributed  group 
decision  makers  to  share  their  UI. . . 

-  Improve  overall  group  decision  quality  through  the 
effective  integration/weighting  of  shared  UI  items . . . 

-  without  imposing  an  excessive  cognitive  burden 

•  This  Year: 

-  Expand  conceptual  model  designed  to  address  these  issues 

-  Experimental  verification  of  selected  model  concepts 

-  Interface  with  EWALL  technologies 
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Expected  Final  Products 
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•  A  collaborative  software  suite  that  includes 

-  Simplified  Knowledge  Elicitation  technologies 

for  adding  (tagging)  individual  information 
evaluation  parameters. 

-  Group  database  browsers  for  finding/ sharing  UI 

-  Ewall-type  interface  for  using  this  tagged 
information  to: 

•  Assist  a  team  member  make  a  decision/recommendation 

•  Assist  the  group  in  reaching  final  overall  consensus 
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Decision  Process  Flow  Model1 


I  need  a  decision  /recommendation  regarding. . . 
■Consider  factors  A,  B  &  C  in  your  decision 


•A 


Ted^S 


DCODE  Research  Issues 

"I  I  need  a  decision  /recommendation  regarding. . . 

I _ Consider  factors  A,  B  &  C  in  your  decision 
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fei  ^  t 

How  can  we  assist  the  individual  in 
reaching  an  overall  recommendation? 
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A? 

1  b? 


Research  Questions 

How  can  we  assist  the  individual  in 
evaluating  and  scoring  a  criterion? 
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The  analyst  sorts  and  structures  incoming  information 
items  (belongs  to  A  or  B  or  C) 

The  analyst  assigns  his  subjective  evaluation  to 
specific  information  content  (a  knowledge  elicitation, 
“tagging”  process).  Parameters  might  include: 

-  Importance 

-  Information  impact  (positive  or  negative) 

-  source  credibility 

-  timeliness,  etc. 
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Item  has: 

Source 

URL 

Time  date 
Title 

Summary 

Body 


h  In 


~ 


&  Netscape 


.COm/WORLD 


GO 


SERVICES 


Newswatch 


On  CNN  TV  Transcripts  Headline 


Nicaragua  to  study  banan 
pesticide  claims 

Allegedly  exposed,  workers  want  U.S.  corp< 


Wednesday,  November  20.  2002  Posted:  10:06  AM  EST  (1506  C 


Technology 
Science  8c  Space 
|  Health 
Entertainment 
Travel 
Education 
Special  Reports 


Rent  DVDs 
Online 

Try  Netflix  for  FREE! 


MANAGUA.  Nicaragua  (AP)  -- 
The  Nicaraguan  government 
agreed  Tuesday  to  form  an 
interagency  commission  to 
study  the  plight  of  thousands 
of  banana  workers  who  claim 
they  were  exposed  to  harmful 
pesticides  while  working  at 
U.S. -owned  fruit  plantations  in 
the  1970s. 

The  move  came  as  dozens  of  former 
banana  workers  protested  outside  the 
U.S.  embassy  in  Managua,  the 
nation's  capital,  demanding  a  solution 
to  the  workers'  lawsuit  seeking  U.S.  $5 


But  It  needs: 

which  category...? 
Importance  (hi-lo) 
Impact  (Pos/Neg) 
Credibility  (good/bad) 
Timeliness  (new/old) 


More  than  2.000  former  banana 
workers  match  Tuesday  in  Managua  to 
demand  compensation  for  exposure. 

Story  Tools 


Want  mor 
articles  lit 
this  onel 

Enter  a  keywoi 
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Each  item  has  two  types  of 

information 


iisrx.vw 
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ID/Tracking  and 
Content  parameters 
(no  real  disagreements) 


Subjective,  decision 
relevant  information 
an  aly  sis/evalu  ation . . . . 
(disagreements  expected) 
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Each  information  item  (“card” 
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ID 
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1 

y 
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1 
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Proposed  knowledge  elicitation 
schema  for  analysis  of  information  items 
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Worst 

Lowest 

Least 

Most  Negative 


Best 

Highest 

Most 

Most  Positive 


Goal:  Minimum 
Cognitive  Burden 


t 


Average 

Neutral 

Middle 

Indifference 


Good  end 
of  scale 

Cannot  determine 
Unknown 
Review  Later 


u!T„tK; 


The  knowledge  elicitation 
“tagging”  process 


“The  Rangoon  Times  reports  unconfirmed 
rumors  that  the  Minister  of  the  Interior  has 
resigned  today  after  the  tragic  collapse  of  the 
newly  constructed  bridge  at. . 


Criteria:  Political  Stability 


Importance: 

Impact: 

Credibility: 

Certainty: 

Timeliness: 


99 

•  • 

99 
•  • 

J _ 

99 

•  • 

J _ 

1 

99 
•  • 

99 

•  • 

Save 


v 


Systems  Center 
San  Dieqo 


22 


UTT„tK; 


Saved  Information  item  #  PS3 


(can  also  be  viewed  as  EWALL  “card”; 
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ID 


Political  Stability  item  #PS3| 

L 

Impor 

Impct 

Cred 

Cert 

Time 

“The  Rangoon  Times  reports 
unconfirmed  rumors  that  the  Minister 
of  the  Interior  has  resigned  today 
after  the  tragic  collapse  of  the  newly 
constructed  bridge  at 

Information 

Analysis 


Short  Descriptor 
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E3  Drawings 


Text 


Color 


HEADING 
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Internet  Sites 
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Computer  Files 


E  Executables 
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VIEW 


INFORMATION  BAR 
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Date  /  Time 
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Votes  (#) 
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Ownership 


Priority  (#) 


TRANSFER 


EWALL  OPERATIONS 
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Distnbution 
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Storage 

<D 

Analysis 

EWALL  TOOLS 

Holding  Area 
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Trash 

FILE  SYSTEMS 


EXTERNAL  DEVICES 
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Shared  Displays 
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Portable  PS’s 
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Handhelds 
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Remote  Storage 

*Paul  Keel,  MIT 
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How  can  we  assist  the  individual  in 
evaluating  and  scoring  a  criterion? 

Analyst  uses  interface  to  EWALL  technology  to 

evaluate  each  criterion 


rTTca^Talilit 


Political  Stability  Items 


The  EWALL  algorithms  uses  the  tagged  information 
to  sort,  rank,  link  (whatever  the  user  specifies) 
the  items  into  a  meaningful  framework  for  analysis 
and  final  criterion  evaluation/scoring 
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Analyst  uses  EWALL  technology  * 
to  evaluate  criterion 
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Political  Stability  Summary 
(5  items  for  each  of  five  criteria) 

#1  2  3  4  5 


Cred 

Cert 

Time 


* 

s 

S3 

Political  Stability  Rating: 
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How  can  we  assist  the  individual  in 
reaching  an  overall  recommendation? 

Same  as  before  except  now  the  items/cards  represent 
his  evaluation  score  assigned  to  each  criterion 

Scoring  of  four  selection  criteria 


Political  Stability 
Cost  (use  of  assets) 
Duration  of  Ops 
Local  Security 


Individual  Recommendation:  NO 
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How  do  we  give  participants  access  to  uniquely 

held  information? 


My  PS  Items 


1  Political  Stabil 

ity  item  xxx  | 

Impor 

Impct 

Cred 

Cert 
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Other  analyst’s  PS  items 


1  Political  Stabili 

ty  item  xxx  ] 

Impor 

Impct 

Cred 

Cert 

Time 

“The  Rangoon  Times  reports 
unconfirmed  rumors  that  the  Minister 
of  the  Interior  has  resigned  today  after 
the  tragic  collapse  of  the  newly 

constructed  bridge  at. . 

1  Political  Stabili 

ty  item  xxx  |J 

i 

Impor 

Impct 

Cred 

Cert 

Time 

“  ASDJKDAS  ASDKJDAS 

ADLADAS  AD  DKDSAKDSAD 
DSALDAS  ,SASA  SA  SAJAS  ASJSA 
ASDJAS  S 

1  Political  Stabili 

ity  item  xxx 

Impor 

Impct 

Cred 

Cert 

“  ASDJKDAS  ASDKJDAS  ' 

ADLADAS  AD  DKDSAKDSAD 
DSALDAS  ,SASA  SA  SAJAS  ASJSA 
ASDJAS  S 

Political  Stability  item  xxx 
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Cred 
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Cert 
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l 

Time 

Political  Stability  item  xxx 


— 
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Cert 
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Political  Stability  item  xxx 
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Impct 
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Cred 
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Cert 

Time 

“  ASDJKDAS  ASDKJDAS 
ADLADAS  AD  DKDSAKDSAD 
DSALDAS  ,SASA  SA  SAJAS  ASJSA  | 
ASDJAS  S 


Send  me  non-duplicative  items  rated  Important 


&  Tecb*°Vc 


My  Full  Array  of 
Information  Items 
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Political  Stability  item  xxx 


Impor 

Impct 

Cred 

Cert 
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DSALDAS  ,SASA  SA  SAJAS  ASJS 


Political  Stability  item  xxx 
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ASDJAS  S 


Political  Stability  item  xxx 


Impor 

Impct 

_  Cred 

_ Cert 


“  ASDJKDAS  ASDKJDAS 
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ASDJAS  S 


Shared 


My  Unique 


Political  Stability  item  xxx 


Impor 

Impct 

Cred 

Cert 
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DSALDAS  ,SASA  SA  SAJAS  ASJS 


Political  Stability  item  xxx 


“  ASDJKDAS  ASDKJDAS 
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Political  Stability  item  xxx 
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Political  Stability  item  xxx 


Political  Stability  item  xxx 
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DSALDAS  ,SASA  SA  SAJAS  ASJSJ^ 
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Unique  from  analyst 


“  ASDJKDAS  ASDKJDAS 
ADLADAS  AD  DKDSAKDSAD 
DSALDAS  ,SASA  SA  SAJAS  ASJSJ^ 
ASDJAS  S 


funique  from  analyst  4 


With  Transfer  of  Unique  Items: 
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Political  Stability  item  xxx 


— 

Impor 

Impct 

Cred 

Cert 

“  ASDJKDAS  ASDKJDAS 
ADLADAS  AD  DKDSAKDSAD 
DSALDAS  ,SASA  SA  SAJAS  ASJSf. 

itinn.iitin  n 


The  item  content  and  scoring 
is  transferred... 


1  Political  Stabi 
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Impct 
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Cred 
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Cert 

Time 

but  scoring  can  be 
modified  by  recipient. 


Political  Stal  ill 
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Political  Stability  item  xxx 
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I  Unique  from  analyst  3 


|  Unique  from  analyst  4 
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ALL  the  Analysts  have  the 
same  array  of  PS  items 
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Political  lability  item  xxx 
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THIS  is  what  differs 
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At  this  point... 
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•  There  has  been  NO  collaboration  between  participants 

-  exchange  of  UI  was  assumed  to  be  an  automated  database 
search  process  on  tagged  information  items 

•  Tagging  and  EWALL  technology  have  been  used  to 
make  individual  judgements/decisions 

(When  in  an  asynchronous  mode,  a  large  amount  of  the 
information  is  processed/analyzed  independently) 

•  But,  the  final  GROUP  recommendation  requires. . . 

-  collaboration 

-  consensus  building 
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Collaboration  is  via 
shared/networked  EWALL  display 
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•  If  the  UI  exchange  process  has  been 
successful,  all  the  participants  are  using  the 

same  information  items 

•  Differences  in  scoring  is  on  the  individual 
interpretation  of  the  information  items 

•  Individual  interpretation  is  reflected  in  the 
tags  assigned  by  each  participant  and 

-  can  serve  as  a  focus  for  group  discussion 

-  forms  a  framework  for  consensus  building 
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Reaching  Group  Consensus  on 
Criterion  Scores 
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Five  Decision  Criteria  (“A”),  Five  Participants  (“P”) 
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How  can  we  assist  the  Group  in 
reaching  an  overall 
consensus/recommendation? 

EWALL  items  now  become  the  group  criteria  scores 

Group  scoring  of  decision  criteria 


Al:  Political  Stability 
A2:  Cost  (use  of  assets) 
A3 :  Duration  of  Ops 
A4:  Local  Security 
A5:  Infrastructure 


Consensus  Overall  Recommendation: 
Mild  Endorsement 
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Decision  Flow  Model  (Revised 


Systems  Center  \ 
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Decision  Flow  Model  (cont.) 

EWALL  Technology 
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Systems  Center 
San  Dieqo 


Each  participant  has 
a  uniform  pool 
of  items 
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^^Collaboration  &  Consensus  Building 

(PS  Criterion) 

Networked  EWALL  Computer  Display 


fcV.  ',Vf< 


V 
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All  PS  Items  PS  Criterion 

Scores (3) 


political  bTaolit 


political  bTaolit 


political  staolit 


political  siaDlit 


political  sraDlit 


political  5iaDlit 


pi 


P2 


P3 


■meal  staDlit 


iiticai  siaDlit 


■meat  5taDlit 


political  Stablit 


J  Item 


Ifcimcai  Stablit 

IT  I  scores 

■  (pi) 


■meal  StaDlit 


■mcai  Maoiit 


■meal  MaDiit 
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Group  Overall  Decision 


WIsTi  I'.KW 


V 
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San  Diego 


Networked  EWALL  Computer  Display 
Group  Criteria  Overall 


Scores 

Scores (3) 

|  iiticai  Maojit 

p  ^^JHca^faf5|it 

|  |itica^faE|it 

mca^raDiit 

P2 

■lticaTStaDlit 

^^JT!ca^faE|ii 

Jmca^rab  lit 

P3  1  1  1 

Completed  Experiments 


wis/i  mm 


v 
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•  Two  experiments  completed  in  FY02 

•  Primary  focus  was  on  what  type  of  information  the 
subj  ect  would  request  to  reconcile  differences  with 
the  “group” 

•  Subjects  focused  on  maximum  discrepancies  with 
“other  group  members”  vice  attempting  to  move 
mid-point  scores  to  a  consensus 

•  One  experiment  showed  significant  preference  for 
looking  at  scores  higher  than  their  own  (i.e.  more 
positive) 

-  second  experiment  showed  opposite  result  40 


Planned  Experiment 


WIsTi  I'.KW 


V 

Systems  Center 
San  Diego 


•  Structure  &  Tagging  Experiment 

•  Design/software  completed,  pilot  subjects 
tested 

•  Experiment  is  on-line,  internet  based 


http://64.66. 5. 34:8080/dcode 


•  Expect  to  use  Ss  at  NPS,  run  Jan/Feb. 
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Experimental  Design  Overview 


Systems  Center  \ 
San  Diego 


Original  Pool 
of  Items 


iiiismi 


UBI 

pjvT" 


Phase  I 


Composite  pool 
of  items  (UI+SI) 


SStl  BfiEHSTCfli 

Kiln 


Phase  II 


Research  Question: 

Does  the  manner  in  which  these  are  displayed, 

influence  the  accuracy  of  the  revised  assessment? 


Structrure  &  Tag  Experiment 


WIsTi  I'.KW 
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•  Subject  sees  12  information  items 

-  4  Criteria,  3  information  items  per  Criterion 

-  Scores  all  the  criteria,  plus  an  overall  socre 

-  Then  sees  1 0  new  items  which  are  UI  from  other 

members  (e.g.  automated  browser  search) 

•  UI  creates  hidden  profile  situation,  i.e.  Negative  changes  to 
Positive  with  information  pooling 

-  Then  scores  his  constructs  again  after  seeing  the  UI 

•  Three  Presentation  Modes  (Between  Ss  variable) 

-  Randomized  lists  of  the  1 2  and  the  12+10  items 

-  The  12  and  12+10  item  lists  are  grouped  (but  not  tagged) 

-  The  1 2  and  12+10  item  lists  are  grouped  and  tagged 
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&  Ted>*°VC 


List  Condition  (First  12) 


Systems  Center  \ 
San  Diego 


"...The  major  ports  of  I  slandia  are  less  than  three  days  sailing  from  our  major  disaster  recovery  supply  center  at  Naha. 
Naha  also  has  three  supply  ships  that  are  permanently  loaded  to  be  used  in  quick  response  assistance...." 

"... I  slandia  Security  Forces  are  used  to  working  in  small  independent  groups,  and  as  such  have  never  really  developed 
expertise  in  larger  scale  command  and  control  operations  such  as  this  camp  would  required.  There  is  some  question  as  to 
whether  they  will  be  able  to  effectively  perform  amission  of  this  large  scale...." 

"  ...The  Huan  Construction  Company  announced  last  month  that  it  had  completed  all  the  work  on  Aurgon,  Islandia's  major 
airport.  The  major  impact  of  this  construction  is  that  the  airport  will  now  be  able  to  handle  even  the  largest  commercial  and 
military  jet  aircraft..." 

"...  .Our  Ambassadortolslandiabeenatthatjob  longer  than  any  other  in  the  re  gion,  and  has  a  very  good  relationship  with 
all  levels  of  the  I  slandia  government.  This  should  assure  that  any  negotiations  between  us  and  I  slandia  would  go  very 
quickly  and  without  any  major  difficulties.  .." 

"...  .A  detaile  d  rep  ort  on  b  oth  the  primary  and  s  e  c  ondary  ro  ads  in  I  slandia  indie  ate  s  that  within  and  b  etwe  en  all  the  maj  or 
cities,  almost  all  of  the  primary  roads  are  four  lanes.  The  vast  majority  of  the  secondary  roads  are  good  quality  two-lane 
highways...." 

"...In  the  past  eighteen  months,  a  number  of  large  manufacturing  plants  have  opened  in  I  slandia.  Asa  consequence,  the 
unemployment  rate  is  quite  low.  Some  companies  have  reported  difficulties  in  attracting  a  local  lab  or  force,  and  some  have 
been  forced  to  bring  in  foreign  workers...." 
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Scoring  the  information 


WIsTi  I'.KW 


V 

Systems  Center 
San  Diego 


Please  assign  scores  to  each  of  the  following  factors 


INFRASTRUCTURE 


LABOR  POOL 


IMPACT  ONUS 


r  2  r  3  r  4 
Neutral 


r  2  r  3  r  4 

Mid 


<“5  r(  r  7 

Yes 


n  r  7 
High 


r  2  r  3  r  4 

Neutral 


f~  2  f  3  r  4 
Mid 


r  5  r  6  r  7 

Yes 


O  5  r  6  r  7 
High 


c  2  r  3  r  4 

Neutral 


r  2  r  3  r  4 

Mid 


r  j  r  6  r? 

Yes 


r  5  r  6  r  7 

High 


C  2  C"  3  r  4 
Neutral 


r  2  r  3  r  4 

Mid 


r  j  r  j  r  7 

Yes 


r  s  c  6  r  7 

High 


Overall  Rating 


f'l<~2(~3C'4('5f'6(~7 


Score 


Confidence 


Overall 
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0^“ 

Generic  KE  design  was  not  used 


■  ?? 

•  • 

•  KE  design  had  not  been  finalized  during 
software  development. 

•  Felt  a  finer  degree  of  granularity  might  be 
needed  to  highlight  differences  (7-pt.  Scale 
vs.  5-pt.  Scale) 

•  Only  one  tagging  parameter  (impact,  pos- 
neg)  was  used  for  tagged  condition. 
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List  Condition: 

UI  Items  (appended  to  end  of  list) 


wis/i  mm 
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"  .  ..Last  year  a  World  Bank  survey  listed  Islandia's  labor  force  as  having  the  highest  literacy  rate  of  any  country  in  the 
region.  This  has  been  true  for  the  last  nine  annual  surveys,  and  it  attributed  to  Islandia's  emphasis  on  giving  every  child 
access  to  a  free  education...." 

"...Members  of  Islandia's  Security  Forces  are  hand  picked  by  government  agencies.  As  such,  the  members  feel  strong 
allegiance  to  the  central  government.  Their  loyalty  to  the  government  and  their  immediate  compliance  to  government 
orders  and  directives  has  never  been  an  issue.  They  do  as  they  are  ordered  by  the  Islandia  government...." 

"  ...  The  Security  Force  proposed  for  the  camp  is  of  adequate  size  but  there  is  concern  that  should  another  emergency 
appear- and  with  the  oncoming  monsoon  season,  that  is  a  strong  possibility- the  Security  Forces  would  not  be  able  to 
cover  both  the  camp  and  the  other  disaster.  In  that  case,  the  Islandia  government  would  almost  assuredly  give  top  priority 
to  the  new  disaster,  leaving  the  camp  with  insufficient  protection...." 


|| 

First  12 


"...The  IslandiaNews  reported  the  two  Northern  Provinces  experienced  a  six-hour  loss  of  electric  power  during  the  night. 
This  is  the  seventh  time  this  month  that  there  has  been  a  prolonged  power  loss,  indicating  that  Islandia  has  still  not 
solved  its  chronic  problem  of  failing  to  have  a  reliable  power  grid  system  in  any  part  of  the  country  other  than  the  Capital 
region..." 

"....The  most  recent  issue  of  the  Asian  Times  carries  an  excerpt  from  a  Doctors  Without  Borders  report  highlighting 
the  serious  sewerVwaste  disposal  conditions  they  found  in  rural  Islandia.  Many  of  the  large  sewer  pipes,1  drains  were 
found  to  he  old  and  leaking ,  and,  if  not  repaired  quickly  by  the  government,  these  medical  personnel  felt  there  could  he 
serious  health  issues...." 


"...A  cultural  tradition  in  Islandia  is  that  each  worker  is  given  a  single  two  month  vacation  each  year,  which  he  can  take 
any  time  during  the  year.  All  the  employee  has  to  do  is  give  the  employer  a  one  week  notice  before  going  on  vacation. 
This  has  caused  unexpected  and  disrupted  work  shortages  when  several  employees  elect  to  take  the  vacation  at  the  same 
time...." 


Next  10 

V 


"....With  the  Islandia  government  supplying  the  bulk  of  the  Security  Forces,  we  should  have  to  send  only  a  minimum 
number  of  our  uniformed  military'  to  the  camp .  This  would  therefore  hare  virtually  no  effect  on  the  ability'  of  our 
command  to  respond  to  some  new,  unplanned  crisis...." 
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Scoring  the  information 
again  after  seeing  UI  items 
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Which  UI  items  were  most 

important? 


wis/i  mm 
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Reference  Window 


*na 


Second:  |  ~^1 

Third:  |  z\ 


1 .  "....The  most  recent  issue  of  the  Asian  Times  carries  an  excerpt  from  a  Doctors 
Without  Borders  report  highlighting  the  serious  sewer/waste  disposal  conditions 
they  found  in  rural  Islandia.  Many  of  the  large  sewer  pipes/drains  were  found  to  be 
old  and  leaking,  and,  if  not  repaired  quickly  by  the  government,  these  medical 
personnel  felt  there  could  be  serious  health  issues...." 

2.  "...A  cultural  tradition  in  Islandia  is  that  each  worker  is  given  a  single  two 
month  vacation  each  year,  which  he  can  take  any  time  during  the  year.  All  the 
employee  has  to  do  is  give  the  employer  a  one  week  notice  before  going  on  vacation. 
This  has  caused  unexpected  and  disrupted  work  shortages  when  several  employees 
elect  to  take  the  vacation  at  the  same  time...." 


3.  "....With  the  Islandia  government  supplying  the  bulk  of  the  Security  Forces,  we 
should  have  to  send  only  a  minimum  number  of  our  uniformed  military  to  the 
camp.  This  would  therefore  have  virtually  no  effect  on  the  ability'  of  our  command 
to  respond  to  some  new,  unplanned  crisis...." 

4.  "...Pacific  Telecommunications  reports  that  it  has  failed  to  come  to  agreement 
with  the  government  of  Islandia  regarding  establishing  more  telephone  lines  in 
the  rural  areas  of  the  country.  Aside  from  the  Capital  region,  Islandia  ranks  as 
one  of  the  worst  countries  in  the  region  in  terms  ofphones  per  capita  and  has  no 
cellular  phone  system  at  all...." 

5.  "....Our  intelligence  reports  no  terrorist  or  anti-government  activity  in  Islandia 
over  the  past  decade.  The  government  is  strung  with  no  militant  opposition  par-ties. 
This  should  ensure  that  our  troops  stationed  in  the  camp  are  in  minimal  danger  of 
outside  attack...." 


- - The  10 

UI  items 

We  would 
expect  he  selects 
the  hidden  profile 
reversal  items 
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Grouped  Condition: 
First  12 


ttffMUI,,,,; 


SPAWAR 

v 
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***  INFRASTRUCTURE  *** 

"...The  Huan  Construction  Company  announced  last  month  that  it  had  completed  all  the  work  on  Aurgon,  Islandia's  major 
airport.  The  major  impact  of  this  construction  is  that  the  airport  will  now  be  able  to  handle  even  the  largest  commercial  and 
military  jet  aircraft..." 

"....A  detailed  report  on  both  the  primary  and  secondary  roads  in  Islandia  indicates  that  within  and  between  all  the  major  cities, 
almost  all  of  the  primary  roads  are  four  lanes.  The  vast  majority  of  the  secondary  roads  are  good  quality  two-lane  highways..." 

"  ...  The  Islandia  News  reported  the  two  Northern  Provinces  experienced  a  six-hour  loss  of  electric  power  during  the  night.  This  is 
the  seventh  time  this  month  that  there  has  been  a  prolonged  power  loss,  indicating  that  Islandia  has  still  not  solved  its  chronic 
problem  of  failing  to  have  a  reliable  power  grid  system  in  any  part  of  the  country  other  than  the  Capital  region..." 


***  LABOR  POOL  **** 

"  ...In  the  past  eighteen  months,  a  number  of  large  manufacturing  plants  have  opened  in  Islandia.  Asa  consequence,  the 
unemployment  rate  is  quite  low.  Some  companies  have  reported  difficulties  in  attracting  a  local  labor  force,  and  some  have  been 
fore  e  d  to  bring  in  foreign  workers ..." 

"...Last  year  a  World  Bank  survey  listed  Islandia's  lab  or  force  as  having  the  highest  literacy  rate  of  any  country  in  the  region. 
This  has  been  true  for  the  last  nine  annual  surveys,  and  it  attributed  to  Islandia's  emphasis  on  giving  every  child  access  to  a  free 
education...." 

"....Worker  productivity  in  Islandia  has  been  difficult  to  assess  due  to  their  record  keeping  methods  and  governmental  control 
over  the  release  of  the  data.  However,  a  recent  survey  of  multinational  companies  with  plants  in  Islandia  indicates  that  they  feel 
their  plants  in  Islandia  are  always  among  their  most  productive..." 


Items 
grouped 
by  criteria 
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*  Grouped  Condition: 

UI  items  appended  to  end  of  category  list1 


***  INFRASTRUCTURE  *** 

"  ...  The  Huan  Construction  Company  announced  last  month  that  it  had  completed  all  the  work  on  Aurgon,  Islandia's  major 
airport.  The  major  impact  of  this  construction  is  that  the  airport  will  now  be  able  to  handle  even  the  largest  commercial  and 
military  jet  aircraft....11 

"...  A  detaile  d  rep  ort  on  b  oth  the  primary  and  s  e  c  ondary  ro  ads  in  I  slandia  indie  ate  s  that  within  and  b  etwe  en  all  the  maj  or  citie  s, 
almost  all  of  the  primary  roads  are  four  lanes.  The  vast  majority  of  the  secondary  roads  are  good  quality  two-lane  highways..." 

"...The  I  slandia  News  reported  the  two  Northern  Provinces  experienced  a  six-hour  loss  of  electric  power  during  the  night.  This  is 
the  seventh  time  this  month  that  there  has  been  a  prolonged  power  loss,  indicating  that  I  slandia  has  still  not  solved  its  chronic 
problem  of  failing  to  have  a  reliable  power  grid  system  in  any  part  of  the  country  other  than  the  Capital  region..." 

“...A  recent  magazine  article,  in  response  to  the  upcoming  monsoon  season,  noted  the  deplorable  history  that  Islandia's  Dept. 
ofRoads  has  in  repairing  the  road  andbridge  damage  inflicted  by  the  monsoon  rains .  While  the  road  system  itself  in  Islandia  is 
considered  quite  adequate ,  repairs  to  damaged  roads  and  bridges  often  takes  several  weeks ,  causing  excessive  delays  or  leaving 
areas  totally  inaccessible...." 

"....The  most  recent  issue  of  the  Asian  Times  carries  an  excerpt  from  a  Doctors  Without  Borders  report  highlighting  the 
serious  sewer/waste  disposal  conditions  they  found  in  rural  Islandia.  Many  of  the  large  sewer- pipes, 'drains  were  found  to  be  old 
and  leaking,  and,  if  not  repaired  quickly  by  the  government,  these  medical  personnel  felt  there  could  be  serious  health 
issues...." 


"....Pacific  Telecommunications  reports  that  it  has  failed  to  come  to  agreement  with  the  government  of  Islandia  regarding 
establishing  more  telephone  lines  in  the  rural  areas  of  the  country.  Aside  from  the  Capital  region,  Islandia  ranks  as  one  of  the 
I  rvorst  countries  in  the  region  in  terms  of  phones  per  capita  and  has  no  cellular- phone  system  at  all...." 
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Condition  (First  12) 
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Systems  Center 
San  Dieqo 


***  INFRASTRUCTURE  *** 


THIS  ITEM  IS  RATED  AS  POSITIVE  - "  ...  The  World  Health  Organization  recently  cited  Islandia  as  an  outstanding  example  of  a 
nation  addressing  health  issues.  Specifically  noted  was  the  completion  of  their  Potable  Water  Initiative  (PWI),  a  three-year 
program  directed  at  giving  every  village  with  a  population  of  over  150  either  a  large  storage  supply  of  potable  water  or  access  to 
a  high- output  Reverse  Osmosis  machine  to  produce  the  required  potable  water  supply...  " 

THIS  ITEM  IS  RATED  AS  POSITIVE  -  "...A  detailed  report  on  both  the  primary  and  secondary  roads  in  Islandia  indicates  that 
within  and  between  all  the  major  cities,  almost  all  of  the  primary  roads  are  four  lanes.  The  vast  majority  of  the  secondary  roads 
are  good  quality  two-lane  highways...  " 

THIS  ITEM  IS  RATED  AS  NEGATIVE  -  "...  Pacific  Telecommunications  reports  that  it  has  failed  to  come  to  agreement  with  the 
government  of  Islandia  regarding  establishing  more  telephone  lines  in  the  rural  areas  of  the  country.  Aside  from  the  Capital 
region,  Islandia  ranks  as  one  of  the  worst  countries  in  the  region  in  terms  of  phones  per  capita  and  has  no  cellular  phone  system 
at  all...." 


***  LABOR  POOL  **** 

THIS  ITEM  IS  RATED  AS  POSITIVE  -  "  ...  Last  year  a  World  Bank  survey  listed  Islandia' s  labor  force  as  having  the  highest 
literacy  rate  of  any  country  in  the  region.  This  has  been  true  for  the  last  nine  annual  surveys,  and  it  attributed  to  Islandia's 
emphasis  on  giving  every  child  access  to  a  free  education...  " 

THIS  ITEM  IS  RATED  AS  NEGATIVE  -  "  ...  Past  regional  conflicts  in  the  area  have  caused  hostile  feelings  among  the  different 
cultures  and  countries.  Some  of  the  evacuated  refugees  will  be  fromTammni,  a  country  that  in  the  past  has  had  poor  relations 
with  Islandia.  Islandia  workers  may  not  be  willing  to  work  at  the  camp  if  they  perceive  that  they  are,  in  part,  helping  T  ammni 
residents..." 

THIS  ITEM  IS  RATED  AS  POSITIVE  -  "  ...  The  salary  and  benefits  paid  to  Islandia  workers  has  been  compared  against  ten  other 
countries  in  the  region.  The  findings  vary  as  a  function  of  the  specific  issue  addressed,  but  in  almost  all  of  the  comparisons, 
Islandia  workers  are  in  the  lower  half  in  terms  of  salary  and  benefits  paid..." 


Items  grouped 
and  tagged  as 
Positive  or 


Negative 


influence  on 
criteria. 
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Tagged  Condition: 

UI  items  appended  to  end  of  category  list1 
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***  INFRASTRUCTURE  *** 

THIS  ITEM  IS  RATED  AS  POSITIVE  -  "...  The  World  Health  Organization  recently  citedlslandia  as  an  outstanding  example  of  a 
nation  addressing  health  i:  sues.  SpeciJ  ,c  ally  noted  was  the  completion  of  their  Potable  Water  Initiative  (PWI),  a  three-year 
program  directed  at  giving  every  vmage  with  a  population  of  over  150  either  a  large  storage  supply  of  potable  water  or  access  to 
a  high- output  Reverse  Osmosis  machine  to  produce  the  required  potable  water  supply.  .." 

THIS  ITEM  IS  RATED  AS  POSITIVE  - "  ...A  detailed  report  on  both  the  primary  and  secondary  roads  in  Islandia  indicates  that 
within  and  between  all  the  major  cities,  a  most  all  of  the  primary  roads  are  four  lanes.  The  vast  majority  of  the  secondary  roads 
are  good  quality  two-lane  highways..  ." 

THIS  ITEM  IS  RATED  A  3  NEGATIVE  -  "  ..Pacific  T ele communications  reports  that  it  has  failed  to  come  to  agreement  with  the 
government  oflslandiar  garding  establis:  ingmore  telephone  lines  in  the  rural  areas  of  the  country.  Aside  from  the  Capital 
region,  Islandia  ranks  as  me  of  the  worst  ountries  in  the  region  in  terms  of  phones  per  capita  and  has  no  cellular  phone  system 
at  all...." 

THIS  ITEM  IS  RATED  A  S  NEGATIVE  -  "  ..A  recent  magazine  article,  in  response  to  the  upcoming  monsoon  season,  noted  the 
deplorable  history  that  L  landia's  Dept,  of  Roads  has  in  repairing  the  road  and  bridge  damage  inflicted  by  the  monsoon  rains. 
While  the  read  system  ii  ;elf  in  Islandia  i  considered  quite  adequate,  repairs  to  damaged  reads  and  bridges  often  takes  several 
weeks,  causing  excessiv  delays  or  leavir  ;  areas  totally  inaccessible...." 

THIS  ITEM  IS  RATED  A  S  NEGATIVE  -  "  ...The  most  recent  issue  of  the  Asian  Times  carries  an  excerpt  from  a  Doctors 
Without  Bordets  report  ughlighting  the  serious  sewer/waste  disposal  conditions  they  found  in  rural  Islandia.  Many  of  the 
large  sewer  pipes/drairu  were  found  to  be  old  and  leaking,  and,  if  not  repaired  quickly  by  the  government,  these  medical 
personnel  felt  there  cou]  Ibe  serious  hea]  h  issues...." 

THIS  ITEM  IS  RATED  A  S  NEGATIVE  -  "  ...The  Islandia  News  reported  the  two  Northern  Provinces  experienced  a  six-hour 
loss  of  electric  power  du  ing  the  night.  T  is  is  the  seventh  time  this  month  that  there  has  been  a  prelonged  power  loss, 
indicating  that  Islandia  I  as  still  not  soke  its  chronic  problem  of  failing  to  have  a  reliable  power  grid  system  in  any  pari  of  the 
country  other  than  the  C  ipital  region...." 
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Experimental  Hypotheses 
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•  List  condition 

-  Will  not  detect  the  hidden  profiles 

-  Least  accurate  scoring  of  criteria 

-  Low  confidence 

-  Longest  completion  time 

•  Tagged  condition 

-  Will  detect  the  hidden  profiles 

-  Most  accurate  scoring  of  criteria 

-  Highest  confidence 

-  Shortest  completion  time 

•  Grouped  Condition 

-  Middle  range  between  List  and  Tagged  conditions 
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Additional  FY03  Work 

Optimized  KE  techniques/displays:  Five  Criteria 
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What  information  is  required, 
how  should  it  be  elicited? 
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Compatibility  with,  leveraging  of 
EWALL  technologies 
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Interface/displays  for 
group  consensus  building 
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